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FOREWORD 



Work Unit REALISTIC was undertaken by the Human Resources Research Organiza- 
tion to provide information concerning demands for reading, listening, and arithmetic 
skills in selected military occupational specialties, and to provide information relevant to 
the matching of personnel literacy skills to job demands for such skills. This report 
presents information bearing on both of these objectives, with emphasis on the first. 
Information bearing on the second objective is discussed m separate reports of research 

The research was conducted at HumRRO Division No. 3, Presidio of Monterey, 

California, where Dr. Howard H. McFann is Director. ,t r» u 

Military support was provided by the U.S. Army Training Center Human Research 
Unit, Presidio of Monterey, California. Successive Military Chiefs of the Unit during the 
researcli project were LTC David S. Marshall, LTC Robert Emswiler, and LTC Ullrich 

The research was performed by Dr. Thomas G. Sticht, Dr. John S. Caylor, and Dr. 
Richard P. Kern, with the assistance of Mr. Lynn Fox, Mr. William H. Burc^aitt, and 
Mrs. Rachel Chisum. Additional assistance was rendered by Dr. Richard McCrady who 
was with the Work Unit for several months and Mr. Donald F. Polden. Military assistants 
from the Human Research Unit were SP5 James Ford, SP4 Donald Enderby, SP5 Richard 
Ferrington, and SF5 Richard Wuerthner. 

Research reported herein as Sub-Unit REALISTIC I made use of data collected by 
HumRRO Division No. 3 Work Unit UTILITY. The design and conduct of Work Unit 
UTILITY were executed by Dr. Robert Vineberg and Dr. Elaine Taylor, with the literacy 
interests of REALISTIC I being appended to UTILITY for that portion of this research. 

HumRRO research for the Department of the Army is conducted under Contract 
DAHC 19-70-C-0012. Training, Motivation, and Leadership reseai'ch is conducted under 
Army Project 2Q062107A712. 



Meredith P. Crawford 
President 

Human Resources Research Organization 



MILITARY PROBLEM 

The ability to perform adequately on many military jobs is determined to s^e 
extent by the individual’s ability to read, listen, or handle arithmetic computations. Th^e 
content and teaching level of many military training courses and the reading level o 
many job manuals and other technical materials presupposes literacy levels as high as_ 
college sophomore or higher. Men who are below that level there ore, can be expected to 
experience difficulty in using instructional material and ma, aals in training for 

performing their military jobs. . - « • iqcc 

The initiation of Project 100,000 by the Department of Defense m 1966 brought 
many men of marginal literacy skills into the Armed Services. This inf mx of men with 
lower-level language and reading skills strains a training system geared to ^ 
platform types of instruction. Classification of these men into jobs for which their 
Lding skms are sufficient is also a problem, since little information is available 
concerning the literacy skill levels suitable for a given job. Furth^more the designation 
of targeted levels of remedial literacy programs, such as Army Preparatory Training, is 
performed blindly, since it is not known what level of reading-language skills is needed 

for adequate job performance. 



RESEARCH PROBLEM 

Taking cognizance of the literacy problems encountered with the Project 100.^00 
personnel mLrower planners initiated HumRRO Work Unit REALISTIC. The obj^t.ves 
were to provide information concerning th' demands for readmg, arithrnetic and Jistening 
skills in MOSs composed of large number, of men, including the new standards ^en 
Project 100,000. Also, information was sought concerning utilizing men of low liter y 
levels through remedial training or modification of job demands. 



METHOD 

Literacy requirements were studied in four Military Occupational Specialties (MOSs): 
General Vehicle Repairman (MOS 63C), Unit and Organization Supply Specialist (MOS 
76 Y), Armor Crewman (MOS HE), and Cook (MOS 94B). Three methocs were used to 

study the reading level requirements of these jobs. , ,■ „i, „ 

(1) Relationships were examined between the reading, listening, and aritl.metic skills 

of job incumbents ai the above MOSs, and the performance of these men on three 
different indices of job proficiency: Job Knowledge tests. Job Sample performance s, 

and Supervisor’s Raiings.* , 

(2) Relationships wdre examined between reading skills of new Army accessions and 

their proficiency on job-related reading task tests constructed from printed matenals 
reported by job incumbents to be used in their work as Repairmen, Supply Specialists, or 

Cooks. 



‘ These tests were constructed and administered under HumRRO Work Unit UTILITY. 
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The materials selected for the reading task tests were identified during inter- 
views with job incumbents in these MOSs. During the interview, information^ was 
obtained concerning a man’s job-related reading, listening (i.e., acquiring information by 
asking others), and arithmetic behaviors. 

(3) Reading level demands of the printed materials identified as in (2) were 
estimated by means of a readability formula, and Uiese levels were compared with the 
reading skill levels of the men in the MOSs. 

Additional v/ork on REALISTIC involved considerable laboratory and literature 
research .'o obtain information of potential use for reducing gaps that exist between an 
individual’s literacy skill level and the demands of his job fnr these skills. Much of the 
laboratory work focused on the study of listening as a substiLulr for reading, and is 
reported in HumRRO Technical Reports. Only the results and implications of this 
laboratory and literature research are discussed in this report,. 



RESULTS 

Concerning literacy demands of jobs, the research indicated that: 

(1) Reading ability was related to job proficiency, measured by performance 
on job-related reading task tests, job knowledge paper-and-pencil tests, and job s,ample 
tests. For iob reading tasks, the reading test score was more highly related to proficiency 
than were scores on the Armed Forces Qualification Te.-jt (AFQT). For the job knowledge 
and job sample proficiency tests, AFQT and reading test scores predicted proficiency 
about equally wei’. 

(2) The teaaing difficulty of printed job materials (except for the Cook MOS) 
far exceeded the reading ability of both high and low aptitude men. 

(3) If the materials were too difficult, men tended not to use them. The 
difficulty level of job reading materials was negatively related to the frequency with 
which they were used on tlie job. 

(4) The greater a man’s reading ability, the more likely he was to use printed 
job materials, even the more difficult materials. 

(5) Performance on job sample performance tests tended to improve when 
manuals wore used during the test. 

(6) Listening ability was related to job proficiency when measured by either 
job knowledge or job sample performance tests. In addition: 

(a) Category IV personnel said they obtained almost three-fourths of their 
information by listening. 

(b) One-fourth of a sample of some 300 men reported that they preferred 
to learn by listening rathei than by reading; among those identified as very poor readers, 
the proportion preferring to learn by listening rose to one-half. 

(c) A sample of high (Non-Category IV) and low (Category IV) aptitude 
men learned prose material as well by listening to tapes as by reading. 

(d) Men interviewed on the job reported many instances of listening for 
information. The frequen :y of listening, relative to reading, was higher for poorer readers 
in MOSs where reading naterials wore most difficult. 

(7) Arithmetic skill was related to reading, AFQT, and job proficiency when 
measured by either job knowledge or job sample tests. While arithmetic skills showed 
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some promise as a predictor of job performance, interviews with job incumbents in the 
Repairman, Supply Specialist, and Cook fields indicated little actual use of arithmetic on 
the job. Overall, demands for arithmetic skUls did not exceed what is ordinarily taught in 

the first six grades in school. ..... i, o i o 

(8) Minimal literacy demands were found to be highest for Supply Specialist 

(ninth grade level), next highest for Repairman and Armor Crewman (eighth grade level), 

and least for Cook (seventh grade level). 



CONCLUSIONS 

Oi the basis of these results, it was concluded that: 

(1) Greater use of job reading materials, and hence better job performance, 
might be stimulated either by upgrading the reading skills of personnel through literacy 
training, or by reducing the difficulty levels of materials by providing systems-engineered 

job aids, or both. . - ■ i ... 

(a) If remedial training is given with the aim of providing a closer match 

between a man’s ability and job requirements, suggested targeted grade levels of reading 
skills are: Cook, seventh grade; Armor Crewman and Repairman, eighth grade; Supply 
Specialist, ninth grade. Since job requirements differ, remedial training would appear 
most effective when oriented toward a specific job; general non-job-related hteracy 
training, geared as it is at present in the armed services to a uniform fifth-sixth grade 
level, would probably not have much effect on job proficiency. Diagnostic literacy tests 
could be applied to a job area selected by the man prior to his enlistment, both to 
identify minimal enlistment skills, and to determine the adequacy of the slulls for the 
reading level needed in the job area. Upon accession, men deficient m job-related literacy 
skills could be given remedial training, using job-related reading, listening, and arithme ic 
materials, to improve their job proficiency and general military suitability. 

(b) Literpiture research in REALISTIC has indicated that properly 
designed job manuals can enable the less experienced and less literate rnen to achieve 
acceptable job performance, with fewer errors and little or no loss m 

much less training than is usually given. The approach to manual design should ^ task 
analytic” with the manual pretested on the prospective user. If materials were redesi^ed 
to facilitate job performance of men in the MOSs studied in REALISTIC, it should be 
expected that men having literacy skills below those identified could use the material,'!. 
They could, thus, perform more effectively while receiving remedial literacy training to 
permit them to move into higher level jobs in which use of general military publications 

would be necessary. . . 

(2) In view of the fact that the reading test predicts proficiency on job reading 

tasks better than the AFQT does, and identifies what might be done to rernedy a man’s 
skiU level, consideration could be given to replacing, or supplementing, the screenii'g 
instrument (AFQT) with more diagnostic, skills-oriented instruments. Such mstrumente 
could serve both as screening tools and as indicators of how much remedial training a 
man might need in some literacy skill area (reading, arithmetic, listening, writing, 

speaking) to improve his job proficiency and opportunities. . . . , 

(3) If elimination, retention, or promotion of incumbents is largely contingent 
upon job knowledge test performance, formal procedures should be implemented to 
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ensure that men have opportunities to acquire both the job knowledge and the literacy 
skills needed to take the test. Skills in reading, listening (if the test is orally presented), 
arithmetic, or other language /communication may influence test achievement. 

(4) Because of the importance of listening, research is needed to study the 
listening process further and to attempt to improve listening/thinking skills. Such a 
project. Work Ui.\it LISTEN,' is under way at HumRRO Division No. 3 as a follow-up to 
Work Unit REALISTIC. 



' HumRRO Work Unit LISTEN, Development of Automated Programs to Improve Listening Skills 
Required in Army Jobs. 
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Chapter 1 

INTRODUCTION 



In 1966, the Department of Defense announced the initiation of Project 100,000. 
Under this project, the Armed Services began admitting men who were, by previous 
standards, disqualified for service because of low mental aptitude or, in fewer cases, 
deficient health standards. 

With the implementation of Project 100,000, it was anticipated that there would be 
an influx of many men of marginal literacy into the services. This was confirmed by 
statistics (1) which indicated that 31% of a sample of 46,000 Project 100,000 men read 
at or below the fourth grade level of ability. Some 68% of these low aptitude men read 

at or below the sixth grade level of ability. 

Because of the low literacy skills of many of the Project 100,000 men, there was 
concern among manpower specialists that many of these “new standards” men might be 
assigned to jobs wherein the demands for literacy skills might far exceed the abUity of 
the men, and possibly lead to job failure. To reduce this possibility, information about 
the literacy demands of military jobs was needed. Such information is valuable in making 
decisions regarding classification of men in job areas, in setting minimal levels required in 
adult basic education to render a person qualified for a given job, and in determining 
whether the literacy requirements of jobs might be altered through such measures as 
redesigning reading materials or providing audio materials. Such redesign of materials 
could possibly reduce entrance requirements in literacy skills so that individuals deficient 
in such skills could be effectively utilized, perhaps while receiving training to improve 
their literacy skills, and thus prepare themselves for higher level job positions. 



WORK UNIT REALISTIC 

To obtain information concerning the literacy demands of military jobs. Work Unit 
REALISTIC was initiated in 1968. REALISTIC is an acronym based upon the three 
literacy skill areas studied — REAd ing, UStening, and ArithmeTIC. The objectives of the 
research were to (a) provide information concerning the demands for reading, listening, 
and arithmetic skills in several riiajor military occupational specialties (MOSs), and 
(b) provide information and suggestions for reducing discrepancies between personnel 
literacy skill levels and the literacy skill levels required by the job. 

To determine the literacy requirements of jobs, we used the concurrent validity 
model, in which both Uteracy skills and job proficiency measures are assessed concur- 
rently, using as subjects only men currently qualified as job incumbents. The literacy 
requirement of a job is determined as the minimum literacy competence associated with 
a specified level of satisfactory job proficiency. If the data contain a range of both 
literacy and proficiency scores, by using this model the degree of relationship between 
literacy and job proficiency can be empirically determined. As a concomitant, this model 
also yields data on the Uteracy skUls characteristic of job incumbents meeting any and all 
criterion definitions of job proficiency. 

However, while the concurrent vaUdity model yields information a^ut the literacy 
skUls of men who are performing their jobs at different levels of proficiency, it provides 
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no information about the literacy skills of those men lost through attrition during the 
selection and training processes or reassigned out of the job. To the extent that literacy 
skills are related to selection, training, and job performance attrition, the concurrent 
validity model will provide underestimates of the true relationships between predictor 
variables (literacy, AFQT) and criteria. 

For establishing the literacy requirements fc'- job performance, the concurrent 
validity model is sufficient. For the larger manpov.'or management purpose of selecting 
and training qualified job incumbents, the concurrent validity model provides only partial 
data and requires, in addition, a full attrition study of all the potential job performers 
who never attained or could not retain job incumbency. 



DEFINITION OF LITERACY 

In applymg the model described in the preceding section, relationships between 
literacy skill levels of job incumbents and various measures of their job proficiency were 
examined. Before this research procedure is discussed in detail, some general considera- 
tion should be given to a definition of “literacy” skill. Most simply, literacy refers to 
reading ability. Most reading experts today, however, recognize that reading is a complex 
of skills and abUities. It is also generally recognized that reading involves the abUity to 
translate printed symbols into auditory symbols, (i.e., into speech) and that, in the early 
stages of reading training, most readers silently speak the words on the printed page to 
themselves. As a reader advances in reading skill, there apj>ears to be less need to 
sub-vocalize the words, so there is a more direct decoding of printed symbols into 

meaning. ... u j- 

The fact that listening ability typically precedes reading ability, and that the readmg 

process is usually built upon a structure of listening ability was of particular concern. It 
was conjectured that some lower aptitude men might not be able to learn a job well by 
reading, but would be able to learn by listening or by demonstration. Their primary 
literacy defect might be one of lack of skill in decoding printed symbolism into speech. 
Thus, listening skill was included in the research as a subcomponent of “literacy.” 

As a matter of secondary interest, and at the request of the sponsor, arithmetic sMl 
was included in the study as a measure of a specific literacy skill, consisting of the ability 
to receive and process information in a highly specific, formal, and rigorous lanpiage. 
Budget and time constraints precluded the study of the expressive literacy skills of 
speaking and writing. 



RATIONALE FOR THIS RESEARCH 

The question may he raised as to why one might expect to find relationships 
between literacy skills and job proficiency. The REALISTIC research was premised on 
the idea that language skills (including reading and listening) are primary me^s of 
learning about job tasks, their requirements, and how to accomplish them. Thus, in the 
formal training programs which precede most jobs, information (vicarious experience) is 
acquired in large part through reading and/or listening. Then, in day-by-day performance 
of the job, some tasks (e.g., reading manuals and forms) are inherently reading tasks, 
while many others, which make no immediate demands on reading skills, may have been 
previously learned by reading, and hence have an indirect requirement for reading. 

Other job tasks, however, may he learned by asking co-workers or supervisors. Such 
tasks, while not demanding reading skills, make a demand on listening skills. Thus, one 
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can expect that, to the extent that job tasks make past or present demands on language 
skills, their successful performance will be related to some degree to measures of literacy 
skill. 

Yet, it must be emphasized that being able to exercise literacy skills to obtain 
information about what to do and how to do it is only one (complex) factor in the set 
of factors (e.g., personality, attitudes) that determine job proficiency. For this reason it 
was not expected that knowledge of literacy skills would account for all, or even the 
greater part of, the differences in job proficiency observed in a broad spectrum of job 
incumbents. 



OVERVIEW OF WORK UNIT REALISTIC 

In pursuing the objectives stated above, research <jn REALISTIC was undertaken in 
three Work Sub-Units (Figure 1). The effects of literacy on job proficiency were 
examined by studying relationships among the reading, listening, and arithmetic skills of 
personnel and their performance on one or more of four different measures of job 
proficiency. As Figure 1 indicates, the indices used included proficiency on job-related 
reading tasks, proficiency on hands-on Job Sample performance tests, proficiency on Job 
Knowledge paper-and-pencil tests, and proficiency as measured by supervisor ratings. 

Relationships between literacy skills and proficiency on the job performance test, 
knowledge test, | and supervisor ratings were studied under Sub-Unit REALISTIC I.‘ 
Relationships between literacy skills and performance on job-related reading tasks were 
explored under Sub-Unit REALISTIC II. Figure 1 reflects the activities used to identify 
job-related reading tasks and provide information concerning how a man’s reading ability 
interacts with the reading difficulty level of job reading materials to influence the extent 
of their use. 

An additional direction of effort in REALISTIC III was to identify information that 
provides a basis for reducing gaps which may exist between an individual’s literacy skill 
level and the demands of the job for these skills. This activity involved some experi- 
mental work, primarily in regard to listening, but was mainly based on a literature survey. 
As indicated in Figure 1, reduction of discrepancies between the skills people have and 
the skills needed for the job was to be considered either by modifying the person 
through literacy training, or by modifying the job demands through the substitution of 
listening for reading demands, and the redesign of job and training aids to mc>ke them less 
demanding of literacy skiUs. Work on REALISTIC III is described in more detail in works 
on listening in relation to aptitude (3^,^. 

In the Chapters which follow, research dealing with how reading ability, with its 
attendant language component, may exercise a pervasive influence on a man’s information 
seeking and processing on the job is described first (Chhpter 2). Following the examina- 
tion of the Oii-the-job reading practices of men in selected jobs, consideration is given to 
the relationships observed between reading ability and proficiency ph job reding tasks 
(Chapter 3). Chapter 4 describes he relationships between literacy skiiis and the three job 
proficiency indices studied in Sub-Unit REALISTIC I. Chapter 5 discusses methods for 
reducing gaps between literacy demands of jobs and literacy skills of personnel. Finally, 
Chapter 6 summarizes the research findings and their implications for personnel selection, 
classification, and training, and for design of job aids to facilitate job performance. 



* The work or REALISTIC I was performed in conjunction with HumRRO Work Unit UTILITY and 
shared the same subjects (cf. 2). 
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Organizational Chart for Work Unit REALISTIC 
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Chapter 2 

LITERACY AND ON-THE-JOB READING BEHAVIOR 



The data in this chapter show how the difficulty level of printed job materials and 
the level of the reader’s skill may interact to influence the extent of use of job reading 
materials. Data also show how reading ability relates to the practice of asking and 
listening to others for job-related information, and to the extent and nature of usage of 
arithmetic on the job. These data were collected under REALISTIC II in the course of 
identifying job reading materials for use in the construction of job reading task tests. 

Three Army jobs were studied: General Vehicle Repairman (M0S63C), Unit and 
Organizational Supply Specialist (MOS 76Y), and Cook (MOS 94B). These jobs were 
selected for study because they represent a range of literacy demands as determined from 
the job requirements stated in AR 611-201. ‘ These jobs were included in both 
REALISTIC r and Work Unit UTILITY; thus, there is a comprehensive base of knowledge 
about the jobs and about the men working in them. 



THE STRUCTURED INTERVIEW 

Job-related reading materials were identified by means of a structured Interview 
administered to men in the target MOSs at their job locations. In addition to personal 
data, such as name and unit, information was obtained regarding the use of listening and 
reading sources in obtaining job-relevant information, and the nature and frequency of 
use of arithmetic skills. 

To identify use of listening sources the man was asked to give five instances when 
he had asked somebody for job-related information in the last month or so. Probing was 
continued until he could not think of any more instances. 

A similar procedure was employed to identify use of printed materials or reading 
sources. The man was asked to give five examples of times, during the past month or so, 
when he had used printed materials in connection with performing a job. In each case he 
was asked to describe the job he had been performing and to tell what information he 
had been seeking when he went to the printed material. He was asked to get the manual, 
locate the exact page he had used, and then show the interviewer the specific parts of the 
material he had used in obtaining the desired information. This process was repeated until 
either five instances had been described or the individual could give no more; in any 
event, he was not pressed for examples beyond five in number. 

A similar procedure was used to elicit comments about the use of arithmetic. An 
attempt was made to get at least five citations of the use of arithmetic. The type of task 
performed using arithmetic was noted, and the type of arithmetic (i.e., addition, sub- 
traction, etc.) was determined from the nature of the tasks reported. 



^ Department of the Army. Manual of Enlisted Military Occupational Specialties, Army Regula- 
tion 611-201, Washington, April 1966, with amendments. 
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Table 1 

Descriptive Data on Men Interviewed in 
Each of the Three Job Positions 







AFQT 


Reading Grade Scores^ 


MOS 


N 


Mean 


Standard 

Error 


Mean 


Standard 

Error 




Supply Specialist 

High Reading Level (9.0+) 
Middle Reading Level (7-8.9) 
Low Reading Level (4-6.9) 



11 


73.8 


5.0 


9 


36.0 


3.2 


10 


17.4 


3.0 


30 







Repairnan 

High Reading Level (9.0+) 
Middle Reading Level (7-8.9) 
Low Reading Level (4-6.9) 



25 


45.2 


4.4 


34 


33.3 


2.7 


26 


24.7 


2.5 


85 







11.2 0.4 
7.7 0.1 
6.3 0.2 



10.1 0.2 

7.9 0.1 

6.1 0.1 



Cook 

High Reading Level (9.0+) 
Middle Reading Level (7-8.9) 
Low Reading Level (4-6.9) 



15 

16 
17 
48 



63.6 
35.1 

24.7 



3.8 

3.6 

4.1 



11.2 

8.0 

5.4 



■Survey of Reading Achievement, Junior High Level, California Teat Bureau. 



0.4 

0.2 

0.4 
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subgroup. Because of the high correlation of AFQT and reading, the effects of AFQT on 
reading ability were not removed by either statistical or design procedures, as doing so 
would, in effect, nullify the results of the study. Thus, in the analyses which follow, the 
low reading groups (4-6.9) consist primarily of men with AFQTs from 10-30 (AFQT 
Mental Category IV), while the men in the higher reading groups have AFQTs of 31 or 
greater (AFQT Categories III, II, and I). 



DIFFICULTY OF JOB-RELATED PRINTED MATERIALS IN 
RELATION TO READING ABILITY 

/ 

One way to assess the reading requirements for a job is to examine the reading 
materials used by men on the job. If, according to the current, formal job specifications, 
a job incumbent is expected to be able to read certain publications, then an evaluation of 
the difficulty level of these publications provides an estimate of level of reading skill 
required by the job. Thus, in the present research, one of the questions of interest 
concerned the reading difficulty levels of the reading materials cited by the men as having 
been used in conjunction with their job activities. 

To determine these difficulty levels, copies of the publications cited during the 
interview were obtained. For the Repairman and Cook MOSs, copies of publications 
which were cited five or more times were obtained. For the Supply Specialist MOS, 
where there were few publications cited, the researchers obtained a copy of the most 
frequently cited reference and copies of a number of other publications which were 
reported by supply sergeants to be of importance and in generd use in the supply field. 

The modified Flesch readability formula developed by Fan, Jenkins, and Paterson 
(6) was used to assess the reading difficulty levels of the job publications. The term 
“readability” refers to the comprehensibility of a publication— that is, how easy it is to 
read and understand it. 

The readability analyses are summarized in Table 2, which shows the total number 
of publications and pages sampled in each MOS, the range of reading difficulty levels 
found over all pages, and the average grade level or readability of materials in the MOSs. 



Table 2 

Readability Scores for Publications in Three Army Jobs 



MOS 


Number 

of 

Publications 


Number 
of Pages 
Sampled 


Range of 
Flesch 
Readability 
Levels — SGE* 


Average 

Readability 

SGE* 


Supply Specialist 


11 


64 


8.5 - 16 + 


16 + 


Repairman 


8 


244 


7.0 - 16 + 


14.5 


Cook 


6 


100 


6.U — 14.5 


9.0 



*SGE = School Grade Equivalent 

The average grade level of readability of materials is shown graphically in Figure 2. 
Included also is an indication of the average reading grade level scores of a sample of 
Army personnel working on jobs within each of the MOSs (these data were obtained in 
the course of the work on REALISTIC I which is described later in the report). The 
reading ability data are provided separately for high aptitude (non-Category IV) end low 
aptitu^ (Category IV) men. 
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Readability of Publications and Reading Ability of Job Incumbents 
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Figure 2 



Making the visual comparisons between readability and reading ability suggested in 
Figure 2, it Ls noted that the difficulty level of printed materials for the Supply Specialist 
and Repairman positions exceeds the average reading achievement levels of the high 
aptitude job incumbents by about five to six grade levels. For the Cook job, however, 
difficulty level of the printed job materials and the reading achievement level of the high 
aptitude personnel are almost numerically matched. The low aptitude Cooks present an 
average reading achievement level that falls only about two grade levels below the reading 
difficulty level of their materials. Since the readabUity formula provides a roughly 
accurate index of the difficulty of the materials. Figure 2 suggests that both high and low 
aptitude personnel would experience considerable difficulty in reading and compre- 
hending the Supply and Repairman job materials. 



READABILITY, READING ABILITY, AND READERSHIP 

In the fields of journalism and advertising, the effects of the reading difficulty level 
of materials on the readership oi' the materials has been examined. Such study has 
indicated that, if materials are written at too high a level of (hfficulty, the number of 
people who read them may drastically decline. For this reason it was of interest to find 
out whether the gap between the reading ability of the men in an MOS and the reading 
difficulty level of the job printed materials might influence the extent to which men 
reported the use of job-related reading materials. It was thought that men of poorer 
reading ability might avoid reading material and instead tend to listen or ask for job 
information to a greater extent than the more able readers. 

20 



10 




Data bearing on the readership and extent of Ustening are presented in Figure 3 for 
men of three reading skill levels and for Supply Specialist, Repairman, and Cook jobs. 
The readership index expresses the number of printed citations as a p>ercentage of the 
maximum number possible if each man had given his full limit of five. Thus, a readership 
index of 20 means that the particular group of men produced, on the average, one 
citation of the use of printed material out ol; a possible five requested; an index of 40 
would mean they produced an average of two printed citations out of the five requested, 
and so on. The listening index was computed in the same manner as the readership index 
by presenting the number of listening citations actually reported as a percentage of the 
maximum number of citations possible. 

Citation of Reading and Listening Information Sources by Reading Ability Level 




Statistical analyses (Appendix A) and the data of Figure 3 indicate that for the 
Supply Specialist and Repairman a dear relationship exists between reading abUity and 
the reported use of job related materials— the inore able the reader, the greater me 
reported use of printed materials. For Supply Specialist, readers in the grades 4- . 
reading group gave only 30% of the maximum possible citations of use of printed 
materials, contrasted with aearly 65% of maximum citations for the group of ^adere m 
the grades 9+ range. For Repairman, the readership increased from 30% to 509b of 
maximum as a function of reading abUity. It is notable tlmt the Cook group reported a 
hi^ (>80%) incidence of use of printed materials, and this was mdependent of readmg 
ability. A possible reason for this will be discussed later. 



} 
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The data for listening indicate a fairly constant use of listening over the three level 
groups in each MOS. In the Supply Specialist and Repairman MOSs the "“8 ^ 

SghHy higher than the readership index for the lower rea*ng groups. However, with 
thptjp small erouDs these differences are not statistically reliable. 

The data show a remarkable degree of consistency. In addition, the relationships 
observed among reading ability, readability, readership, and listening appear to foUow a 
S trr ^ Repairman and Supp y 

SiScSlistTob; there is less tendency for readers of any level to use the matenals, 

^^ougli the men of higher reading abiUty will use them he may 

If the job materials are quite difficult, and the man cannot read ve^ well, he may seek 
proportionately more of his information from others, rather than from reading th j 

matei^s these data indicate that better readers are more Ukely to use manuals, the 
ouestrn St 11 remans as to whether the use of manuals aids job perform^ce. Date 
concerning reading abUity, the use of manuals, and job performance were obtemed in 
R F^TuSTIC I (deLribed in greater detaU in Chapter 4). In that research, job incumbei^te 
wert Snister^^^^^^^^ Job Sample tests in which Repairmen actually 

renaired vehicles and Supply Clerks worked in simulated offices fiUing out forms and 
costing equipment. In these Job Sample tests, job manuals were available so that men 
wh^ we ^tested could use the manual if they wanted. Figure 4 shows relationships 

Job Sample Test Performance as a Function of Reading Ability and 
Use of Technical Manuals 
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between reading ability, use or non-use of manuals, and performance on the Job S^ple 
tests for Supply Specialists and Repairmen. It is apparent that men who use manuals did 
better than those who did not use manuals regardless of reading abUity. Furthermore, of 
those who used manuals, performance was better for the more highly literate men. 

The present data on readability, reading ability, and readership across the ^*'^2 
MOSs suggest that greater usage of job reading materials might be stimulated 
improving literacy skills of the men and by the redesign of reading materials. Further- 
more, the data for the Cook MOS suggest that greater gains m readership might ^ 
expected from the redesign of materials than from increasing the literacy skUls of men. In 
this regard, much research on the construction and evaluation of various job performance 
aids has indicated that by foUowing a systems approach, materials printed for the job 
can be designed which greatly improve the effectiveness of job incumbents across a 
spectrum of aptitude. It seems likely that the provision of such aids would also increase 
the utilization of job printed materials. 



READING ABILITY AND USE OF ARITHMETIC ON THE JOB 

Table 3 summarizes the data on the use of arithmetic in these jobs. The Mithmetic 
citations, like the readership and listening index numbers, are stated in terms of percent- 
age of possible citations. Table 3 also indicates the nature of the arithmetic comprising 
the citations made by men. Thus, Repairmen with a reading level of 9+ gave only ^ 
the maximum possible citations of the use of arithmetic. Of these few citations, 5% 
involved the use of whole numbers, another 5%, decimals or fractions, 27%, some systein 
of measurement (i.e., a money system, changing inches to feet, pounds to oimces) ^d 
63%, the use of a measurement tool (such as a mler or a gauge). In the Repairman jo , 
the use of gauges (torque wrenches, air pressure gauges, etc.) represented the primary use 

of arithmetic. . . , » ^ u \ 

Of special note in the Supply Specialist job is the lower incidence of use of whole 

numbers by higher reading abUity men, and their higher frequency of use of a system of 
measurement. The data indicate that the higher reading abUity men tend to work more 



. Table 3 

Use of Arlihmetic as a Function of Reading Ability 

(Use of Arithmetic is in Percent of Maximum Citations Possible) 







Arithmetic 


Percent of Citations Using: 


MOS 


Reading 

Level 


Citations 

(Percent) 


Whole 

Numbers 


Decimal/ 

Fractions 


System of 
Measure 


Measurem ent 
Tool* 



Repairman 



Supply 

Specialist 

Cook 



9+ 


15 


5 


5 


27 


7-8.9 


6 


9 


0 


27 


4-6.9 


9 


8 


0 


0 


9f 


38 


32 


9 


54 


7-8.9 


33 


73 


0 


27 


4-6.9 


28 


71 


0 


29 


9+ 


32 


33 


0 


63 


7-8.9 


29 


35 


0 


65 


4-6.9 


33 


29 


0 


68 



63 

64 
92 

5 

0 

0 

4 

0 

3 







•Ruler, gauge 
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with the money system in the Supply MOS than do the less able readers. The latter In 
turn do mostly simple counting tasks involving whole numbers. The Cook data tear 
out the importance of systems of measurement in the use of recipe books. This was true 

regardless of reading level. .. . • u htu 

OveraU then, these data indicate a fairly low usage of arithmetic in these jobs men 

used, however, the nature of the arithmetic task is likely to be different for each MOS, 
and, in the case of the Supply Specialist, for different reading abUity groups. Thus we 
find that Repairmen use mostly gauges, while Cooks use the measurement sys ems 
involved in preparing recipes. In the Supply Specialist MOS, men of low or a^r^e 
reading abUity do simple counting, and men of higher reading ability woric with the 

money system and accounting. 
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chapter 3 

READING SKILLS RELATED TO PROFICIENCY 
ON JOB READING TASK TESTS 

ThP nrimarv objective of Sub-Unit REALISTIC II was to identify job reading tasks 
and to evLateL relationships of performance on job reading task tests to performance 

Twas fir^ necessary to define and identify job 
readiL A job rSung task was defined as a task in which a person obtems (or 

^i!:r:freaLT.:;rs 

who attempts to use the reading matter to obtam the job information. t hv 

With the foregoing task definition in mind, job reading materi s were u ^ tr, 

means of a structmed on-the-job interview. As mentioned earlier, each man f ^®^nth 

ritP five instances in which he had used printed materials in his work in the last month 
of so He ^ Tso asH^^ locate the printed material, show the mtemewer the exact 
Lfand section he had used, and teU what kind of information he had been s^kmg. 

With this information, it was possible to subsequently obtain co^^^^^^ 
materials cited as being used on the job by the men in the three MOSs (Supply ^Peci^is , 
7 «v Rortairtnan Cook 94B) With these materials in hand, a classification scheme 

TdefiLTE Whtef eX^for -tn of reading materials cited in each MOS could 
be classified in terms of the type of information it displayed. Thu type of information is 
ffeC to aTthefontent tZ of printed material. The classification system used for 

categorizing the various materials is given in Table 4. , a 4 . i.u^ 

T usLg this classification scheme to construct rea^ task tests toe printed 
materials cited by toe men in each MOS were sotted into the six different content 
' catMories. Setting aside Category I (Tables of Content and 
remaining five categories were sotted, independently, by two judgra, who m . 

judgmente a^eed on 87%, 80%, and 96% in the Repairman, Supply Specialist, and Cook 

jobs, respectivdy. materials, however, problems in the classification scheme 

becaiie IpS Wtat should the unit of classification be based upon-^ tontence o 
SS or a major sub-section of a technical manual? iiow should pictonM 
materials be classified? Such questions indicate the difficulties involved in cla^i ying 
S^^ted maten^rTo the present research, we classified maten^s on the b^is o 

K?. m«ior 8^^ Of c pubUcation. Thus a section which gave the procedures for 

filling Lt a form was classified Procedural Directions, even though there may have been 
standards and specifications given in the context of the matenal. . , , j a. oKnnt 

to ad"to identitytog job reading materials the mtervmw 
the nature of the information a man had been seekmg when he ured the 
other words, the kinds of questions men working on the job addressed to the job g 

materials. 
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Table 4 



Definition of Content-Type Categories 



1. Tables of Content and Indexes: 

Content designating the location of information within 
a publication. 

2. Standards and Specifications: 

Content setting forth specific rules or tolerances to 
which task procedures or the completed product must 
conform. 

3. Identification and Physical Description: 

Content attempting to symbolically represent an object 
via an identifying code (stock number, nomenclature) and/ 
or by itemizing its distinguishing physical attributes. 

4. Procedural Directions: 

Content which presents a step-by-step description of 
how to carry out a specific job activity. Essential ele- 
ments are equipment/materials/ingredients to be used, 
and how they are to be used, with presentation organized 
in a sequential step-wise fashion. 

5. Procedural Check Points: 

Content which presents a key word or highly summarized 
version of what should be done in carrying out a task 
rather than how it should be done. This content differs 
from the content classified under Procedural Directions 
in that it assumes the user knows how to carry out the 
steps once reminded that the step exists and/or reminded 
of the decision factors which determine whether the step 
is required. 

6. Functional Description: 

Content which presents an operating (cause and effect, 
dependency relationships) description of some existing 
physical system or subsystem, or an existing admin- 
istrative system or subsystem. 



From this information, and copies of the printed materials cited as being used on 
the job by the men in the three MOSs, job-related reading t^k tests were constructed. 
These tests represented the most frequently mentioned reading material content types, 
and required the man being tested to seek the kind of information from the materials 
which job incumbents reported seeking. Three separate tests were constructed, each using 

job-specific, MOS materials. • _i.- 

The general nature of the reading task tests is shown in Figure 5, using a portion 
(the Index) of the Cook Reading Task test. As indicated, questions abcut the job reading 
material were presented on the right side of the test booklet and the job-related reading 
materials were presented on the left side of the booklet. This construction was similar for 
all subtests in the job reading task test for each job. 
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Figure 5 




Table 5 lists the subtests in each job reading task test. The variety in the three job 
reading task tests reflects the variety of different content types cited by men in the 
different jobs. Also given is the difficulty level, in terms of the modified Flesch 
readability formula (5), of both the job material and the test material, where it was 
possible to compute the readability index. In all measurable cases, the difficulty level of 
the materials exceeded that of the test questions. 

Table 5 



Content Types and Difficulty Levels of Job Reading Task 
Test Materials and Test Questions 



Job 


Test 


Content Type^ 


Reading Difficulty Level^ 


Job Material 


Test Questions 


Repairman 


A 


1 


NA 


8.5 


(MOS 63C) 


B 


2 


NA 


8.5 




C 


4 


14.5 


8.5 




D 


4 


NA 


8.5 




E 


4 


14.5 


11.0 




F 


6 


16+ 


NA 




G 


5 


14.5 


8.5 


Supply Specialist 


A 


1 


NA 


6.0 


(MOS 76Y) 


B 


2 


NA 


8.5 




C 


3 


NA 


7.0 




D 


4 


16+ 


11.0 




E 


5 


8.5 


7.0 


Cook 


A 


1 


NA 


5.0 


(MOS 94B) 


B 


4 


NA 


7.0 




C 


4 


7.0 


6.0 




D 


4 


8.5 


6.0 



^Readability levels in school grade equivalents (see text). 
**Content Types follow the numbering in Table 4. 



READING ABILITY RELATED TO READING TASK PERFORMANCE 

Relationships of general reading ability to performance on the job reading task tests 
were evaluated by administering the Job Reading Tests (JRT) and a Standardized Reading 
Test (SRT) (described in Chapter 4) to several hundred Army inductees at the Reception 
Center at Fort Ord. In administering the JRT, the directions were read aloud and the 
example was worked through by the examiner, proctor, and the subjects together. The 
number of subjects per testing session ranged from 25 to 45. 

Table 6 shows the relationships of performance on the JRT to both Ihe SRT and 
AFQT measures. AFQT related equally closely (r=.65) to the job-specific reading material 
of the JRT and to the more general reading content of the educationally oriented SRT. 
There is similarly a notable consistency in the higher level of relationship (mean r=.77) 
between the general reading measure (SRT) and the job-specific reading test (JRT) in 
each of the three MOSs. 
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Table 6 



Correlation Coefficients Showing Degree of Relationship 
of Reading Ability and AFQT to JRT Performance 



Job-Related Reading Task Test! New Army Personnel 



Test 


Repairman 


Supply Specialist 


Cook 






r 


N 


r 1 N 


r 


N 



Standardized 
Reading Test® 
AFQT 



.76 200 .78 222 .77 193 

.66 199 .65 221 .64 191 



aThe correlation between AFQT and Standardized Reading Test scores 
for these personnel is .65. 



Figure 6 shows the percentage c.? men tested at a reception station who are at or 
bove three different criterion levels of performance on the JRT m relation to the 
StyT^rof the men as detennined by the SRT m ptovdes an .nd.cat.on of 

^e level of reading skills required to perform job reading tasks. 

The vertical broken line in each section of Figure 6 designates a reading skiU 
n a IpvpI freouentlv referred to as functional literacy (a considerable mcrease from the 
0 Ide levd established in World War II). The data for the Cooks 

iovf that 100% of the people who read at the eighth grade 

eore on the Cook reading teat whether ihrs cnterion ^ 

orrprt For ti e ReDairman reading tasks, however, of those reading at the 8.0 y 

>0% would he'espected to surpass the 70% correct criterion level. For the 

asks, the situation worsens, since only 20% of persons mth a rcadmg l«el 

)e expected to get 70% correct. Thus there are clear differences m the readmg 

leederiOT men Lt trained in the job to use the printed matenals from the three MOSs. 

rhis is discussed in the next section. 



ieadmbility of materials and reading task performance 

The foregoing analyses substantiate the data on the difficulty of 
arlier in this report, where it was indicated that the readmg difficulty ® Pf ^ 

material was higher than that of the Repairman, which in turn was more difficult than 
hat of the Cook (Figure 2). The data for job reading task performance (Figure 6) appear 

0 substantiate the readability data, in that the Cook's ““ 

lifficultv than the Repairman’s, which in turn is less formidable than that of Supply. 

S material, then, the need for advanced madmg skills 

ppeal to iTgreatesTfor the Supply Specialist job, with tl» Reparman and 

ta that order. If one were to consider a functional r^tog level to be one 
it which 80% of the readers would be expected to get 70% of Ihe job rea.^g s i ms 
“then tctiond Btemcy for Cooks would be in range of grade leveU 
7 0 — 79 for Repairmen. 8.0 -8.9, and for Supply Specialists, 12 

It is* likely however, that through Advanced Individual Trainmg m an MOS are^ 
=ku.iL«o^r”-ing of reJant t»chnicsd te^, «Ki 
naterials will permit less able readers to use the job matenals more satistacto.ily. 

> Aaditton.ia.ta bearing on <hi.probl.m.K being collected under HumRRO Work Unit READNEED. 
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Percent Scoring At or Above the Criterion Scores on the Job Reading Task Tests 
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Chapter 4 

LITERACY IN RELATION TO JOB KNOWLEDGE, 
JOB PERFORMANCE, AND SUPERVISOR RATINGS* 



In Chapter 3 relationships between performance on job reading task tests and on a 
standardized reading achievement test were examined. For Repairman (MOS 63C), Supply 
Specialist (MOS 76Y), and Cook (MOS 94B), correlation coefficients for job reading task 
performance and performance on the standardized reading test were fairly large. This was 
not unexpected, inasmuch as both tests were explicitly designed to tap reading skill, wth 
the purpose of identifying levels of reading skills needed to adequately utilize job reading 

materials. 

This chapter is directed toward the examination of relationships between three 
literacy variables (reading, listening, arithmetic) and three indexes of job proficiency 
which were not designed to tap reading skills, at least not in a direct manner. The job 
proficiency indexes include; job knowledge (paper-and-pencil tests), job performance 
(hands-on job sample tests), and supervisor ratings. 

It was anticipated that reading skill would be more highly related to the paper- 
and-pencil tests than to the job sample performance tests. It was also expected that 
substantial relationships might be found between hands-on performance tasks and literacy 
skills because both require some degree of language facility. However, in the case of job 
sample performance, it was recognized that (as data reported in Chapter 2 indicated) men 
frequently learn and perform job tasks by watching ^d talking wth others. In such 
cases, listening is a more relevant language reception skill than reading. Hence a listening 
test was included in the battery of literacy tests. As mentioned earlier, we dso included 
an arithmetic test in the literacy battery to assess ability in the use of a highly formal 
language system. 



RESEARCH SUBJECTS 

The subjects of tiiis research worked in the three MOSs previously considered 
(Repairman, Supply Specialist, and Cook). In addition, men in a fourth MOS, Amaor 
Crewman (MOS HE), were studied. Men in all four of these MOSs were being studied 
under another HumRRO research program. Work Unit UTILITY,^ and the REALISTIC I 
(see Figure 1) literacy measurement requirements were appended to that effort. In 
keeping with the design for Work Unit UTILITY, subjects for REALISTIC I were hi^ 
(AFQT > 30) and low (AFQT < 30) aptitude men who were paired with regard to the 
amount of time they had worked on the job. However, for the purposes of REALISTIC, 



I 






* The psychometric and job proficiency data used in this chapter were obtained under Hum RRO 
Work Unit UTILITY, to which study the literacy measures of REALISTIC were appended in the inte rests 
of efficiency of data collection. Our indebtedness to the UTILITY staff, and in particular to Dr. Robert 
Vineberg and Dr. Elaine Taylor, is gratefully acknowledged. Of course, responsibility for the data analyses 
and interpretatioru of this report rests with the authors. 

’Detailed description of the UTILITY rationale, procedures, subjects and test materials is given in 
references 2, 6, and T. 
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the only importance of this pairing procedure was that the lower aptitude men were 
over-represented in the sample. For this reason the figures presented for literacy skill 
levels should not be construed as parameters of the general Army population. 

The four MOSs which were studied were selected on a variety of criteria: 

(1) They represent a range of types of job skills. 

(2) They are high density jobs to which a substantial portion of low aptitude 

men are assigned. x _i. • 

(3) Except for Armor Crewman, these Army jobs have close counterparts m 

the other services as well as in the civilian society. 

The nature of these jobs is apparent: Cooks prepare food in accordance with a 
master menu and detaUed recipes; Supply Specialists requsition and issue supplies and 
maintain records— primarily clerical tasks; Repairmen diagnose and repair motor vehicles; 
and the Armor Qewmen operate the tank and its weapons systems-a substantially 

procedural set of tasks. • • u 

These jobs differ widely in the number of job-related reading tasks involved m job 

performance, as well as in the literacy and arithmetic requirements for entry. For each, 
the training program contains a large amount of reading material and, for each, extensive 
manuals are available as part of the job equipment. 



LITERACY AND OTHER PREDICTOR MEASURES 

Reading — To assess reading skills, the Survey of Reading Achievement, Junior High 
I^vel (grades 7-9) developed by the California Test Bureau was used. This test was 
chosen because (a) it provides a composite index of reading skills expressed in school 
grade equivalents; (b) grade placement norms span a wide range of abilities (grades 
4.0-14.5), a necessary characteristic when testing men whose AFQT scores range from the 
10th to ’the 99th percentUe; (c) it has a testing time of 40 minutes which could oe 
accommodated within the Work Unit UTILITY testing schedule. 

Arithmetic— Arithmetic skills were measured by use of the Survey of Arithmetic 
Achievement, Junior High Level, CaUfomia Test Bureau, a companion test to the reading 

instrument and selected for the same reasons. 

Listening— Listening skills were measured by a non-standardized experimental test 
constructed for this project. It consists of three short descriptive passages presented orally 
on tape. After each passage, 12 questions, calling for information contained in the 
passage, are read aloud twice, and after each question the subject is required to write his 
single word or short phrase answer. When the passages are regarded as reading material, 
they fall at the 6th, 7th, and 14th grade level of reading difficulty. 

AFQT-The Armed Forces QuaUfication Test (AFQT) is a measure of trainabUity 
which is used as a “screening” measure for acceptancfe into all our armed sei^ices. It 
consists of 25 multiple-choice items in each of four areas: Verbal, vocabulary; Arithmetic, 
simple word problems; Shop Mechanics, matching line drawings of tools and mechanical 
equipment according to function; and Pattern Analysis, or form perception. Corrected for 
guessing, scores from the four sections are pooled and converted to percentile ranks. The 
Project 100,000, “New Standards” men, mentioned in Chapter 1, faU primarily in the 10 
to 20 percentile range. 



JOB PROFICIENCY MEASURES 

The job proficiency measures were developed by the Work Unit UTILITY staff in 
conjunction with Army content experts. There were three different measures of job 
proficiency (see reference 1_, for a more complete description of these measures). 






Job Knowledge-The job knowledge measures were multiple choice tests (rangi g 
from 75 to 96 items in length) for different job areas. These tests v^re ^ 

samples of knowledge considered by experienced job occupants to be important 
competently fulfilling the job. The test items were selected 

ance, not to school training. Since these were paper-and-pencil tests, the need for read g 

skills was inherent in the task. i. » ,-acanf fho Wpv tasks in 

Job Sample-Job sample tests were constructed so as to represent the key tasks, in 

term s of ess i^ality, frequency, and urgency, typically performed by men m the jobs^ 

For the most part they do not directly test reading skUls, but they do represent what 

men do in thLe jobs. Also, the tests were not free of verbal demands since the 

instructions u-crc presented orally. Hence the “f "‘"8 “mponent 

inherent in the testing situation, though not necess^dy involved m 1°^ ^sk Pet se 

The tests were administered by trained testing crews, at speciri test sites, using 
full normal job equipment. Cooks cooked on field ranges; Supply Specialists "'orked m a 
fully equip^d mock-up supply room; Repairmen repaired trucks and 
Cremien performed in tanks, simulating only the firing of weapons In short, the job 
imXperfomiance tests were hands-on samples of job tasks performed in a given career 

“‘‘supervisor’s Questionnaire-Each subject was rated by his immediate supervisor ^o 
had been instructed by the Itudy staff that his ratiny would ^ tor msemch 

numoses only and would have no administrative effect on the subject. Ratings were 
Ened on two rating scales: (a) the Standard Army Enlisted Evaluation Report con- 
sisting of five- or six-point scale ratings on 14 attributes such as co^ct, job per orm- 
ance cooperativeness, reliabUity, job knowledge, and responsibility; ^d (b) an 

expekmentel scale on 12 items such as, “Does he need f twfratog 

mosf>” and “If you could, would you just as soon replace him? Since the two rating 

scales correlated .80, scores of the two scales were summed to increase the reliability o 

the supervisory rating measure of job proficiency in this study. 




RESEARCH FINDINGS 

The ultimate objective of the data analysis is to determine how well performance on 
one or more of the predictor variables predicts performance on 

ficLncy measures. First, however, it is informative to note how performance on the 
literacy tests and AFQT was distributed in the groups of men studied m the 

TrMe 7 shows number and percentage of men at various levels on the y 

tests and the AFQT. These figures are for the four MOSs combined.. It is ^ 8 

w dthou^ half the total sample was from AFQT Mental Category IV, less than 
lit read below the 6.0 grade level, and fewer than 18% performed 

beL the 6 Hade level in arithmetic. Also, although 25% of the persons m our sample 
^ad AFQT in the 1-20 range, only 2% of the su^cts read ^^ow the 4.9 ^ 

level This contrasts with Department of Defense data Q) which in^^ated that, out 
^.Obo “new standards” men (primarily men with AFQTs m 10-20 range) tested at 

the Armed Forces Entrance Examination Centers, some ^ ® ‘ ^ ^ 

Using 31% as the expected proportion of men to ^ found m the 10-20 AFQT m 
who have reading scores below 4.9, we would expect 123 of our sample of 398 men with 
AFQT scores of 10-20, to score below 4.9. Instead, as Table 7 mdicates, ^ly 31 men 
out of 1564, including 398 lower Category IV men, read in the 4.9 or 
Apparently, the very low reading, lower mentel Category IV men do not well sujwe the 

additional screening imposed by BCT, AIT, and early job literacy 

low readers are, in large part, missing from our sample, relationships between lite y 
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Table 7 



Distribution of Subjects by Literacy and AFQT Level 

(Percent) 



Reading 



Score* 


N 


% 


4-4.9 


31 


2.0 


5-5.9 


108 


6.9 


6-6.9 


228 


14.6 


7-7.9 


328 


21.0 


8-8.9 


233 


14.9 


9-9.9 


273 


17.5 


10-10.9 


168 


10.7 


11-11.9 


55 


3.5 


12-12.9 


83 


5.3 


13 & up 


57 


3.6 


Total 


1564 





Arithmetic 



Score* 


N 


% 


4-4.9 


38 


2.4 


5-5.9 


238 


15.2 


6-6.9 


295 


18.9 


7-7.9 


424 


27.1 


8-8.9 


184 


11.8 


9-9.9 


142 


9.1 


10-10.9 


74 


4.7 


11-11.9 


52 


3.3 


12-12.9 


37 


2.4 


13 & up 


80 


5.1 




1564 





Listening 


Score 


N 


% 


1-4 


22 


1.4 


00 

1 


51 


3.3 


9-12 


146 


9.3 


13-16 


308 


19.7 


17-20 


381 


24.3 


21-24 


353 


22.6 


25-28 


230 


14.7 


29-32 


73 


4.7 


33-36 


0 


0 




1564 





AFQT 


Score 


N 


% 


1-10 


24 


1.5 


11-20 


374 


23.9 


21-30 


384 


24.6 


31-40 


190 


12.1 


41-50 


139 


8.9 


51-60 


141 


9.0 


61-70 


131 


8.4 


71-80 


75 


4.8 


81-90 


73 


4.7 


91-100 


33 


2.1 




1564 





* Score=School Grade Equivalent. 



variables and job proficiency should be viewed as attenuated and, in general, as under- 
estimates of the true relationships between literacy and job proficiency. 

Differences among the four MOSs in performance on the literacy tests are P*«sented 
in Table 8. This table shows the percentage of men in each MOS who scored Mow the 
grade 7.0 level on the reading and arithmetic tests, and below a raw score 13 °n the 
^perimental listening test. With regard to reading, the 

the grade 7.0 level in the Supply Specialist MOS is less than one-hdf that of the C^ok 
where fuUy one-third read below the 7.0 level. The Repairman and Armor Crewman jobs 

occupy a midpoint between Supply Specialist and C^ok. ^ ..i, + rpsirlinff 

It is interesting to recall from earlier portions of this report that the rea^g 

materials in the Supply Specialist MOS are most difjult, 

difficult, with the Repairman in between. If men were being assigned to there th«ee jobs 
on the basis of the reading difficulty of the materials, the assi^naent 
reading achievement distributions in Table 8. That is, smee Supp y ^ rpadinc 

difficult reading materials, it would not be desirable to assign many of the poorer reading 
men to that career field. However, since the Cook reading materials are much easier, men 



Table 8 

Men Scoring Below Grade Seven on the Reading ai^ 
Arithmetic Tests and Below Score Thirteen on the Listening Test 



Military Occupational Specialties 



Literacy Test 


Supply Specialist 


Armor Crewman 


Repairman 


Cook 




N 1 Percent 


N 


Percent 


/V 1 Percent 


N 1 Percent 



Reading 

Arithmetic 


65 

118 


16.1 

29,4 


81 

156 


21.0 

40.6 


94 

146 


23.7 

36.9 


127 

151 


Listening 


42 


10.4 


41 


10.7 


20 


5.1 


116 



24 



34 



of lower reading abUities could be assigned to that job. Possibly then, the classification of 
men into these career fields would take into account, at a judgmental level, the reading 



demands of the job. ... e 

A further index of the differences among the MOSs is the performance on the 

listening test. Approximately one-third of the Cooks scored 12 or below on this test, as 
compared to 5 to 10% of the men in the remaining jobs. Differences in terms of 
arithmetic performance are not pronounced, although Supply Specialist ^ain showed the 
fewest men in the lower skUl range. The data of Table 8 suggest that the hteracy 
demands may be greatest for Supply SpeciaUst, about equal for Armor Crei^ an and 
Repairman, and least for Cook. This is consistent with the findings reported earlier in this 

chapter. 



INTERRELATIONSHIPS AMONG LITERACY MEASURES 

Table 9 presents an intercorrelation matrix for literacy variables, selected personal 
background variables, and the AFQT. The data in Table 9 shows that reading, withmetic, 
and AFQT— all tests requiring reading ability— are substantially intercorrelated. This is no 
surprising since one-quarter of the AFQT is vocabulary and one-quarter arithmetic word 
problems. These three tests show similar profUes over the other predictors. Listenmg, a 
verbal but non-reading test, presents a somewhat attenuated version of this pattern. 

With regard to literacy and personal background factors. Table 9 shows positive 
relationships of reading and arithmetic to education in our sample. Age is not correlated 
to any extent with any of the other variables, while the negative coefficients for race 
with the various measures of cognitive skills suggest that English-speaking Caucasians 
score somewhat higher on these language-oriented measures than non-Caucasians or 
Caucasians from homes where standard English may not be the predominant language. 

The relationship indicated by thp correlation between AFQT and reading grade m 
Table 9 is depicted graphically in Figure 7. The height of the solid line at any 
represents the average reading grade level for men at the AFQT level mdicated on the 
baseline. The dotted lines are drawn one standard deviation (1.6 ^de unite, <>« ^h® 
average) above and below the mean score and constitute a band within which two-thirds 
of the scores would be expected to faU. Thus for men with AFQT of 35, the average 



Table 9 



I ntarcorreiations of Prodictor and Salactod Backjround 
Variables Summed for Four Army Jobs 



Variables 


Variables 




1 


1 ^ 




4 


5 




7 


1. Reading 


— 


.69 


.65 


.54 


.30 


-.04 


-.27 


2. Arithmetic 


.69 


— 


.61 


.43 


.34 


-.10 


-.22 


3. AFQT 


.65 


.61 


- 


.47 


.26 


.01 


-.30 


4. Listening 


.54 


.43 


.47 


— 


.12 


.00 


-.22 


5. Education 


.30 


.34 


.26 


.12 


— 


.01 


.12 


6. Age 


-.04 


-.10 


.01 


.00 


.01 




.11 


7. Race® 


-.27 


-.22 


-.30 


-.22 


.12 


.11 





■For computing the correhliono with race, Caucasian was assigned the code 
number of O, and Negro the code number of 1. 
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Relationship Between AFQT and Reading 




Figure 7 

reading grade level was about 8.3. It would be expected that two*thirds of the men with 
that AFQT would read between the 6.7th and 9.9th grade level; of the remainder, 
one-sixth would read below, and one-sixth above these points. 



INTERRELATIONSHIPS AMONG JOB PROFICIENCY MEASURES 

A second group of variables of interest is the job proflciency measures, the job 
knowledge and job sample test scores and the supervisor ratings. The interrelationships of 
these variaUes determine to what extent these measures describe a man's job inroficiency 
in a similar fashion. If tests agree in describing proHciency, men who score high on the 
job knowledge test would be expected to score high on the job sample test and 
supervisor rating. Thus, high correlation coefHcients indicate that men are being ranked 
about the same firom one index of proficiency to another. Low coefficients indicate that 
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tests are ranking men in a different manner, with perhaps Uttle agreernent from test to 
test. This may be evidence that the criterion measures are sampling different aspects of 

^ *^Before the interrelationships among the measures of job proficiency are considered, 
it should be pointed out that, because of the wide range of job proficiency of men in the 
Work Unit UTIUTY sample, and the high correlation of job proficiency to time on the 
job the proficiency scores have been adjusted statistically, usmg regression-based ^ores 
estimated from the logarithm of time on the job, to remove the effects of job 
Hence all of the relationships we discuss are freed of the mfluence of time on the job 

^^*^*^able 10 presents the interrelationships among the adjusted proficiency measures. As 
indicated, the relationships are consistent from one MOS to the next. The coefficiente 
between job knowledge and job sample are higher than those for either toowledge or jo 
sample ^th the supervisor ratings. This suggests that the knowledge and sample teste are 
sampUng more of a common aspect of proficiency than the M^ct of proficiency 
reflected by the supervisor rating. The fairly large (.64) coefficient for the supply field 
between knowledge and job sample reflects the commonality of the 
aspects of the man’s job performance and the paper-and-pencil aspects of the job 
kn^ledge test. Overall then, for all MOSs, the job knowledge and job sample Pertoim- 
ance tests appear to be sampling some similar aspects of job proficiency, whde the 
supervisor rating is largely an independent estimate of job proficiency. 

Table 10 

Intercorrelations Among Adjusted Proficiency Measures* 



Adjusted Proficiency Measures 


Military Occupational Specialties 


Armor Crewman 


1 Repairman 


Supply Specialist! 


Cook 


Knowledge/Performance 


.50 


.50 


.64 


.50 


Knowledge/Supervisor 

Rating 


.21 


.18 


.29 


.27 


Performance/Supervisor 

Rating 


.13 


.16 


.24 


.20 



•See Appendix B. 



INTERRELATIONSHIPS BETWEEN 
PREDICTOR AND CRITERION VARIABLES 

The intercorrelations of predictor and criterion variables, ^th the latter stetoticj^^ 
adjusted to be freed of the effects of time on the job are listed m Table 11. ^e 
supervisor ratings are not reUted to any of the predictor variables to any 
de^.‘ Hence, the prediction of job proficiency w*'®" 

is not feasible with any of these predictor variables and will not be treated further in this 
report. 
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• Sm HumRRO Technical Report 71-1 (2, p. 6) for further dbcuaalon. 
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Table 1 1 



Correlations Between Predictors and Job Proficiency Measures 



MOS 


Predictor 


Job Proficiency Index 


Job Knowledge 


Job Performance 


Supervisor Ratings 


Armor Crewman 


AFQT 


.55 


.37 


.07 




Reading 


.57 


.32 


.06 




Arithmetic 


.49 


.31 


.14 




Listening 


.53 


.29 


.06 


Repairman 


AFQT 


.44 


.32 


.16 




Reading 


.47 


.26 


.17 




Arithmetic 


.39 


.24 


.14 




Listening 


.40 


.38 


.09 


Supply Specialist 


AFQT 


.36 


.37 


.09 




Reading 


.40 


.40 


.10 




Arithmetic 


.34 


.36 


.09 




Listening 


.35 


.42 


.11 


Cook 


AFQT 


.49 


.37 


.15 




Reading 


.56 


.34 


.11 




Arithmetic 


.44 


.31 


.13 




Listening 


.39 


.28 


.07 



The data for job knowledge scores indicate that, across all four MOSs, job knoad* 
edge is most closely related to reading and AFQT, and less closely related to antiinietic 
and listening (with the exception of the Supply Specialist MOS). For job s^ple 
performance, however, all of the literacy variables and the APQT are lower in predictive 
precision. Thus, the data indicate that the literacy variables predict job measures, which 
place a heavy demand on reading, better than they predict job sample data, which 
demand Uttle direct appUcation of reading although general language demands persist. 

The correlation coefficients indicate the overaU relationships among the predictor 
and criterion variables. Perhaps a more meaningful and useful indication of the relevance 
of reading skills for job proficiency may be obtained by considering ^gures 8, 9, 10, and 
U, fhese figures were constructed for each reading level by determining the percentage 
of men in that reading level who scored among the bottom 26% (fourth quarter), 
next-to-the bottom 26% (third quarter), next-to-the top 26% (second quarter), and top 
26% (first quarter) of all ^ose men who took the tests. Thus, each of the bars shows me 
proportion of men at that reading level who scored in each of the four quartern of job 
proficiency. The sum of the proportions in each bar is 100%, which accounts for all of 
the men at a given reading level. 

In each figure, the bar at the bottom indicates the proportion of men that would to 
expected in each quarter (26%) if the correlation of job proficiency and rea^ng was 0. 
Over-representation in any quarter is indicated by a score greater than 26%, whUe 
under-representation is indicated by a score less than 26%. Under- or over-representation, 
especially in the bottom and top quarters, is expected when the literacy-job proficiency 
correlation is greater than rero. In Figures 8, 9, 10, and 11, the numbers within eroh 
division show the proportion of persons at that reading level who are in the designated 

quarter. 
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Reading Grade Level 






Quarter Distributions of Job Knowledge and Perfornuince 
by Reading Grade Level: Armor Crewnian (MOS 11 Et 
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Reading Grade Level 



Quarter Distributions of Job Knowledge and Performance 
by Reading Grade Level: Supply Specialist (MOS 76Y) 

Expected Pattern for f = 0 
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Quarter Distributions of Job Knowledge and Performance 
by Reading Grade Level: Repairman (MOS 630 
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Quarter Distributions of Job Knowledge and Perfornnance 
by Reading Grade Level: Cook (MOS 94B) 
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Fiffure 8 which deals with job knowledge and performance for Armor Crewmen 
shows u!at 98% of the men who read at the 

thp knowledge test Furthermore, 75% were among the best (top quarter) of those ^ 
took^r Ob Lool^ng '^e lowest reading leveh the fipire sho^ that 

i« 9fi% nf mpn in the 4-5 reading level scored above the median on the job 

‘“'‘fetert^rto?h2^‘fi8ures. the probabUities associated with placement in the fourth 

S=i“sS~,^^ 

Armor, .59; Repairman, .50, and Cook, .71. 



READING REQUIREMENTS FOR FOUR MOS 

In principle, aU of the men tested in REALISTIC I were successful 
by vWue of the fact that they were actively serving in a job ^«*ion at *he time of 
testinc This is somewhat analogous to saying that persons who hold a high school 
dioloma can by definition, read at the 12th grade level. In actuality, however, a student s 
reading level is^defined in terms of his reariing test performance relative to other people 

'St. we can descrtbe .de,na,e job proficiency m Urms of the 
relative proficiency of job incumbents. Thus, the data of Figures 8, 9, 10, and 11 h 
KhU™ iTt"Sency for men at each reading level. That is to s^. the rnen m he 
bottom quarter are there because their proficiency was poorer than three-fourths of h 

°‘"“Wi*find however, that because of the significant correUtions of “<* 

nroficiency the different reading levels are disproportionately represented m the four 
Quarters especially in the top and bottom quarters. As the bar at the bottom of the 
fumres indicates, if there were no correlations of reading and job 

J^t to find 25% of the people at each reading level in the bottom qui^r, 25% m the 
thi^ouarter and so forth. This suggests a decision rule for determmmg tee re^mg levd 
to te sS^Tfor The mle is to select tee lowest reading level for whi^ 

representation in the bottom quarter d<^ not exceed 25%, that is, the leve o rep 

sentation that would occur if reading ability did not count (^®)* . ... 

ApVyteg the above rule to the data for Armor Crewman (Figure 8).we fmd tea^ 
for iob knowledge 33% of the people who read at the 7-7.9 level are m the bottom 
Quarter while only 22% of those who read at tee 8-8.9 level are in the bottom qu r. 
ZT L readLriXel of approximately 8.0 grade is indicated as a ^ted readmg level 
tor tee Armor^MOS when^job knowledge is the criterion^a^ation of 
Sta tends to confirm this level. There we find teat 32% of tee 

7-7 9 grade level are among the bottom job performers, while only 17% of the readere at 

7 7.“ g»a . , 1 • *u lit .. ., niiArtor Amin then an 8.0 grade level of reading 

the 8-8 9 grade level are in the bottom quarter. Agam, uien, an o.w gi 
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these levels rank the jobs in the same order as the reading task test criterion (Figure 6). 
That is, Supply Specialist is most demanding of reading skills, the Repairman is next, and 
the Cook is least demanding. Furthermore, this is consistent with the data of Table 8 
which shows 16% of the Supply personnel scoring at the 7.0 reading level or below, while 
23% of the Repairmen and 33% of the Cooks scored at or below the 7.0 grade level. This 
suggests that classification and/or attrition due to job demands has acted to restrict the 
number of poorer reading men in the Supply and Repairman MOSs. 

When similar analyses are performed for the arithmetic (Figures 12, 13, 14, 15) and 
listening data (Figures 16, 17, 18, 19), the indicated level of arithmetic skills for 
Repairmen, Supply Specialists and Cooks is 7. 0-7. 9, whUe for Armor Crewmen a 6. 0-6. 9 
level appears to suffice. For listening, scores in the 16-18 range appear sufficient to 
prevent dis proportionate representation in the lowest quarter of job proficiency for both 
Armor Crewmen and Cooks. For Repairmen and Supply Specialists, scores in the 19-21 
category appear to be associated with “adequate” job proficiency, as defined by repre- 
sentation in the lowest fourth of job incumbents. The increased demand for listening skill 
in the Supply Specialist and Repairman jobs, in contrast to that for the Cook, is 
consistent with the differences in reading requirements for these jobs. This suggests a 
greater need for general language skills in the Supply Specialist and Repairman fields. 

Because of the substantial relationship between literacy measures and AFQT and 
because of the general importance and familiarity of AFQT, parallel analyses of the 
relationship between AFQT, job knowledge, and job sample performance for the four 
MOSs are presented in Figures 20, 21, 22, and 23. Application of the same decision rule 
used to establish the minimum literacy requirements for these four jobs would indicate 
an AFQT level not much less than 30 to insure against overrepresentation in the bottom 
quartile of job knowledge in each of these four MOSs. 

Interestingly, as shown in Figure 7, an AFQT of 30 corresponds in these data to a 
reading grade level of 8.0, which is coming to be regarded as the defining level of 
functional literacy. The same AFQT requirement of 30 seems to be indicated for Job 
Sample performance in the case of Repairman, Supply Specialist, and Cook, while an 
AFQT of 20 ap^iears sufficient to meet the job sample requirements for Armor Crewman. 
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Quarter Distributions of Job Knowledge and Performance 
by Arithmetic Grade Level: Supply Specialist (MOS 76Y) 
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Quarter Distributions of Job Knowladoe and Perfornriance 
by Arithmetic Grade Level: Cook (MOS 94B) 

Expected Pottern for r - 0 



10*10.9 

9-9.9 

8-8.9 

7-7.9 

6-6.9 

4-5.9 



1 2w 


IV III II 1 

Quarter 




53 1 


Job Knowl#<l 90 






^1 Jrm 






T“ 

00 

1 


44 n 








<0 1 






1 19 


14 






1 W. 


r..44 






I- 

_l 1 


;mi] 

1 1 1 — 



100 80 60 40 20 0 

Percent Below the Median Medion 



Percent Above the Median 



11-15.0 

10-10.9 

9-9.9 

8-8.9 

7-7.9 

6-6.9 

4-5.9 



3-Q 



37 



Job Perfermonce 






17 



35 



42 









26 



Dl 






I 34 



1 — 43 \m 


m 


m 


[iD 



j. 



X 



X 



X 



= .31 



X 



X 



100 80 60 40 20 0 

Percent Below the Median Median 



20 40 60 80 

Percent Above the Median 



100 



100 



Figure 15 



38 



48 








Listening Score Level 



i 



Quarter Dlftributiortt of Job Knowledge and Performance 
by Listening Score Level: Armor Crewnian (MOS 11E) 

Ex pected Pottern for r = 0 

t ' iir. mm -l 2s% ] 

IV ill » I 

Quarter 




O 

ERIC 



49 



39 



Listening Score Level 
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Quarter Distributions of Job Knowledge and Performance 
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Chapter 5 

REDUCING DISCREPANCIES BETWEEN 
LITERACY LEVELS AND JOB DEMANDS 

Research described in Chapter 2 focused on the ^erences ^^wwn toe rea^g 
Hirriniltv levels of the printed materials available for perfomung a job, and reading 

d^ribed, toe reading demands of toe materials were considerably above the skill 

“'in'X* ^Pter we diKU» two approaches aimed at closing the gap s*|ch 
re«.IU ^and. of a job exceed the litemcy skiU. of the men a,^^e 

to nerform the job. One approach is to increase the skill levels of personne y ^ 

toe ouT^ is to modify the Uteracy demands of the job by 
materials These approaches are fairly obvious and easy to ver^ize, but 
mentation is neither simple nor inexpensive. Also, they may have entuely ® " 
Tteo^r^th respect to a major criterion for toe implementation of any such action 

the improvement of personnel job proficiency. 
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LITERACY TRAINING AND JOB PROFICIENCY 

With reference to closing the gap between job literacy demantb and personnel 
Utera^ skills through Uteracy training, one of the most fre^mtly 

UmTSiSe traintag of functional flUlerates in the armrf forces du™g WorW Wm IL 

. , . riesieUwMWT rRi in Vii« extensive review of Army trainmg of illiterates m 
Figures reported by Goldberg (8) m his ““"sive reinew o y ™jved Uteracy 

» PO»e.«>d completion of the fourth 

^"‘^^^cr^l^essful performance during World War 11 is frequently cit^ by re^g 

experts (9 10) as an ^^ple of an approach for upgrading the 

iTtier more competent job performers. However, the evaluation of Uie 

0%'Li^^rronTb'^rofici^ r" 

as was attempted was fraught with methodological difficjdt^. .. 

One attempt at such assessment wns made by Hagen and 

el »Kn rpnnrrfq of 1 026 ilUteratcs who entered the Navy during 1944 and who 

999 marginal aptitude men were also stuaed to ^ a companson with men comparable 

to illitcT&tes 1x1^ 9 ?lio did not receive litcrftcy tTBinin^* « • « 

^^Mtelv several methodological Umitations restrict the conclusi^s of to 

study^-rort^MHe^con^^^^^^ 

Shirati^ l^Mnie marginal group Z differed in age and back^und, 
^^T^rTwere inco^lete Sd inconristent. Thus, the 

were not comparable to the control group of margmal aptitude men. Relative, however, 
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f() Ihr normal ar.il maruinal i oiilrol groups, Ihf illiterate's who rt'ce‘i\t'd literacy l^.al^l^^; 
we re sul)se'(|uentlv e harai lerize'd by fewe r [irornolions. lowt-r profu ie-ney ratmj,'s. more 
dise iplinury ae l i.ns, more' lost time' due' to niiseonduet, fe'weT honorable- dise har>,'e‘s, and 
more' VD than the controls. Thus, m this case, literacy training; did not re'sull in 
pr(ukicin« me n comparalile' in job profieie'ncy to marginal aptitude tne'ii who did not 

re'e'e'ive' such train inf?, nor to ncjrmal aptitude men. 

Since' World War II. the Armed Services have had additional opportunities to 
eonduct literacy training, and to evaluate the effectiveness of this training in job 
proficiency. Perhaps the best work is that reported by Goffard (12). In this study, men 
who scored below the 5.0 grade level of reading on a standardized reading achievement 
U'st were given special literacy training. They were graduated from training upon achieve- 
ment of reading at the 4.9 grade level. When compared with comparable control groups, 
that is, marginally literate men who did not receive special training, the experimental 
group did slightly better' on performance and knowledge tests given at the end of basic 
training. Differences were not considered of any practical significance, however, being less 
than 5% in any ceise. 

Additional studies to evaluate the effects of literacy training on job proficiency in 
the Armed Services are reviewed in the book Marginal Man and Military Service, prepared 
by the Department of the Army in 1965 (13). The upshot of these studies is that little 
benefit to job proficiency has been demonstrated from training in basic literacy skills. It 
is important to point out, however, that none of the literacy training aimed at providing 
literacy skills in excess of those represented by a score of 5.0 on standardized reading 
tests. In this regard, the work reported in Chapters 3 and 4 of this report suggests that, if 
training in reading and arithmetic is to have any notable impact on job proficiency, it 
should be extended to at least the grade 7.0 level. These data lead also to the suggestion 
that literacy training should include training in listening as well as in reading. While these 
skills are inter correlated to a degree (r=.55), they are apparently not identical. More will 
be said later about reading and listening. 

Apart from the limited data based on military literacy training programs, no 
additional research evaluating the effects of remedial literacy training on job proficiency 
has been found. To date, the hypothesis that adult-level remedial reading training will 
improve job proficiency remains an hypothesis. But it seems clear that, if such training is 
to have any notable effect on job proficiency, it should be functionally related to a 
particular job field, and it should extend at least to the 7.0 or 8.0 grade level. This is not 
to say that persons with less than seventh or eighth grade reading skills cannot perform 
successfully in a variety of jobs. The data from Chapter 4 clearly indicate that a number 
of less literate men can perform a variety of job tasks. But for the most part, those jobs 
can be, and probably were, learned by being told or by imitation. However, ^ indicated, 
even in these jobs which are high density for Mental Category IV men, the ability to read 
at or above the seventh or eighth grade level was associated with improved job 
proficiency. 

1 

MODIFYING JOB-RELATED READING DEMANDS BY 
SUBSTITUTING LISTENING FOR READING 

The literacy training approach to reducing the gap between personnel literacy skills 
and literacy demands of jobs involves training the men to more closely match their skill 
levels to the skill level demands of the job. An alternative or adjunct to remedial literacy 
training is to reduce a “literacy gap” by means of changing the literacy demands of the 
job. Considerable work has been done in Work Unit REALISTIC to determine the 
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ffasibility of lowering the reading demands of jobs, and job training programs, by 
substituting listening for reading materials. 

The research on listening performed in REALISTIC can be categorized into four 
groupings: (a) studies of listening as a job skill; (b) reading versus listening as learning 
processes; (c) characteristics of listening materials related to learning by listening, and 
(d) research to increase the amount learned by listening in a fixed amount of time. The 
research findings in each of these categories are summarized below. 

Listening as a Job Skill 

As indicated in Chapter 2, to better understand the importance of listening for job 
pierformance, Repairmen, Cooks, and Supply Specialists were interviewed at their job sites 
regarding the extent to which they used reading (manuals, charts, etc.) and listening 
(peers, supervisors) sources to obtain job information. Results mdicated that Repairmen 
and Supply Specialists reading in the grades 4. 0-6. 9 range tended to rely as much (or 
more) on listening sources as on reading sources for job information. Those with better 
reading ability relied less on listening and more on reading sources for job information. It 
was also found that the Cook job involved much less difficult reading materials than the 
Repairman or Supply Specialist jobs, and there was a marked increase in the use of 
reading relative to listening as a source of job information, even for the poorer reading 
men in the Cook job. Thus, the on-the-job interviews indicated that, where reading 
materials are difficult, men of poor reading ability rely heavily upon listening as a basic 
source of job information. 

Further indication of the importance of listening as a job skill was obtained by 
comparing the job proficiency of men of differing listening abilities working as Armor 
Crewmen, Repairmen, Supply Specialists, and Cooks (Chapter 4). Significant correlations 
were obtained in all jobs between listening and job knowledge (paper-and-pencil) tests 
and job sample tests, that is, tests in which men performed actual samples of work: 
cooks cooked, repairmen repaired vehicles, and so forth. With regard to the work sample 
tests, listening skill predicted performance as well as reading skill or AFQT did. Further- 
more, within a restricted range of AFQT scores (0-30), listening ability predicted job 
sample performance as well as the full range of AFQT scores did (2). Thus, listening 
ability appears to *oe a useful dimension for additional screening of Army inductees, and 
it would be useful to explore the use of a listening test for screening purposes. 

Both the on-the-job interviews and the intercorrelations of listening and job pro- 
ficiency have underscored the importance of listening as a job literacy skill. Another 
aspect of concern was a determination of how reading and listening might compare as 
processes for learning job information. 

Reading Versus Listening as Learning Processes 

To compare the relative effectiveness of learning by reading or by listening, we 
prepared the same prose materials as either reading or listening test passages (3). Men of 
low mental aptitude (AFQT scores 10-30) and of higher mental aptitude (AFQT scores 
greater than 30, with a mean of 63) were administered the reading and listening test 
materials, which included materials of grade levels 6.5, 7.5, and 14.5 (Flesch readability 
scores). With reading time limited to that avaUable for Ustening, the men in both aptitude 

groups learned as well by listening as by reading. 

To find out who might prefer to learn by listening rather than by reading, a survey 
of men in several MOSs was conducted (4). The survey indicated that preference for 
learning by listening increased with decreasing reading ability (and decreasing AFQT). 
This is consistent with the results of the on-the-job interviews (Chapter 2), which 
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mdl. aUvl tnat K.-pairm.-ii urul i^upply Sfx-r alisls of 1o«ot r.-adinu abil.ly l. arnod rnurh of 
tlu’ir jof) information by asking; and lust'Tiii'U to others rather than by readinn. 

'l hes<' two studies sufj^est that many tyoes of pro.^* materials could tn- eff«Ttively 
prepared as listening materials so that these who preferred lo learn by listening mi«ht do 

SO 



Characteristics )f Listening Materials Related to 
Learning by Listening 



A series of basic research studies was conducted in REALISTIC 111 to identify 
various factors which affect the comprehonsibility of lisU>niriR maU'rials (3,4). '1 wo of 
these studies indicated that, as materials decreased in readability (assessed by means of a 
roadability formula), they are more difficult to comprehend for both high anu low 
aptitude (AFQT) men. Also, low association strength word pairs (e.g., Lable-river), 
embedded within .sentences were not learned as well as high association strength word 



pairs (e.g., table-chair) embedded in sent^mces. 

Additional work further indicated the importance of linguistic cues to meaning m 
learning by listening. In that research it was found that the listening comprehension of 
high aptitude men increased directly as the listening conditions were varied from no 
message (baseline), to presentation of the message with the words individually spoken and 
presented in random order, to presentation of the words individually spoken but 
.sequenced into a meaningful story, and finally to a normal, prose-like version of the 
message containing inflection, pausation, and rhythmical cues to meaning. 

These studies indicate that, for practical purposes, measures of readability can be 
used as measures of listenability and that materials which are easier to read and 
comprehend will, on the average, also be easier to listen to and comprehend. Thus, 
difficulty level of listening materials can be monitored by a readability formula applied to 

the prose transcript. , • ,, c 

The data on association strength of words within sentences, and on the influence oi 

linguistic cues on the comprehension of materials indicate that listening materials should 
be prepared using most generally known terms, preferably concrete rather than abstract. 
The materiads should be produced with full use of intonational, inflectional, and dramatic 



cues to meaning. , . . , ^ r 

In conjunction with the above research concerning the structural aspects of the 

listening message, considerable attention has been given to the rate of presentation of the 
messages (3,4). In genera), this work has indicated that increasing the rate of speech 
reduces the comprehensibility of the message, although this reduction is usually negligible 
for speech rates up to around 250 wpm (with variability due to the task demands). 
Further research has suggested that the effects of varying the speech rate may be 
different for high mental aptitude and low mental aptitude subjects, depending upon the 
nature of the listening materials. However, this research is insufficient for suggesting 

generalization in this regard. , ■ x 

It is safe to say, however, that, for both high and low mental aptitude subjects, 

increasing the rate of presentation of a message to above 300 wpm will almost certainly 
have a significant detrimental effect on comprehension. Viewed otherwise, however, these 
studies indicate that listening materials recorded in the “normal” range of speech rates 
(e g., 125-175 wpm) can generally be compressed by as much as 20-30% without a large 
loss in comprehensibility. Thus some tradeoff is feasible; a (generally) small loss in 
comprehension can be traded for what may be considerable savings in time. 
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Research to Increase the Amount Learned by Listening in a 
Fixed Amount of Time 



Thr fac*. ‘hat hsU-ning time may siivcd iiy thr time compression of listenmn 
maUTuils. whiU' produc ing only a small loss of comprehcmsicm. sunnesLs the possibility (if 
usinu the time ..avin^s to improve* the overall learning of the materials. Ihis was 
attempte d m a study in which compres.st‘d materials were prosc>nted twice in the* same 
amount of time required to presc-nt the uncompressed me.ssa«c> once- (3). While* siieli 
repe-lition did re*store* the comprehension of compressed mate*rials to normal, for lot 
hiuh and low m *nlal aptitude subjec’ts, overall learning was not improved. 

An additional study (4) attempted to use the time save*d by the compre-ssion proc-e*ss 
to prese*nt additional informati m. with the idea of increasing the total amount U*amc*d in 
a given amount of time. However, neither high nor low mental aptitude .subjects learned a 
greater amount W'hen extended, compressed information was pre.sented, as compared with 
what they learned when less, uncompressed information was presented. Thus, the limited 
research to date has not been successful in using the time saved by the time-compression 
procedure to improve learning over that obtained with non-compressed materials. 

However, shortcomings of this work are many, and more information is needed to 
determine the utility of time saved by the compression of speech. At present, research 
evidence speaks against using this time to simply repeat the entire message or to ac^d new 
information. Possibly, peak comprehension might be improved by selectively repeating or 
elaborating upon the more important aspects of the materials. It might also be useful o 
insert questions into tl;- material to focus attention on upcoming or recently presented 
material. In general, however, the importance of the speech sampling, rate variation 
technique for education and training is the technological feature of being able to expand 
or compress recorded materials. This makes possible the provision of listening rate 
options for a variety of students. 



Summary on Substituting Listening for Reading 

The work done on the feasibUity of substituting Ustening for reading as a means of 
reducing the literacy demands of jobs has been exploratory and fundamental in regard to 
rate-controlled recordings. This work indicates that inen, all along the continuum o 
aptitude, can learn certain verbal prose as well by listening as by reading. Work in 
REALISTIC using rate^ontrolled recordings suggests that, in some cases, both high and 
low aptitude men can learn less difficult materials at rates of speech more than four 
standard deviations above the mean rate of 175 wpm with only moderate losses in 
comprehension. While this may result in as much as a 36% savings in listening time, it 
remains to be demonstrated how the time saved might be used to increase the general 
level of learning. Projected research on HumRRO Work Unit LISTEN (Development o 
Automated Programs to Improve Listening Skills Required in Army Jobs) includes a 
further examination of this problem, as well as research to study the effects of speech 
rate upon particular information processing procedures used by high and low ap i u e 
men. The primary objective of Work Unit LISTEN, however, is to attempt to improve 

listening skills through special training. 



DESIGN OF JOB PERFORMANCE AIDS TO 
REDUCE LITERACY DEMANDS OF JOB 

A job performance aid is any information storage device, such as a manual, 
checklist, or diagram, which is available to assist a man in doing his job. The present 
discussion is concerned primarily with technical manuals as job aids, and the focus is on 
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thf design of such manuals so as to render them more effective. There is considerable 
evidence (H, 15, which indicates that property designed job manuals can enable the 
less experienced and less literate man to achieve acceptable job performance, wnth fewer 
errors and little or no loss in time, and with much less training than has been thought 
necessary. This has been demonstrated for highly technical jobs in which complex 
machines and electronics systems are involved. 

The major principle underlying the success of well-designed aids is that job-related 
information, which is usually stored in a man’s head, is stored in the aid in such a way as 
to maximize the retrievability of the information. Thus, the long training or on-the-job 
experiences which are usually needed for a man to acquire the needed job information 
axe reduced to the time needed to learn to use the job aids. 

It must be emphasized, however, that it is not the mere storage of information in 
the aid rather than in the man’s head which accounts for the effectiveness of the aid. 
Rather, it is the nature and n. inner of storage that is of crucial importance. For instance, 
a technical manual may contain needed information, but this information may oe stored 
in such a way that the user is not able to work with it. In this regard, the research 
reported in Chapter 2 is relevant. That research indicated that while the technical 
manuals in the Repairman and Supply Specialist jobs do present job-relevant infomation, 
they are written at a level well above the reading ability of the men within the job, and 
hence there is a low level of usage of the materials. In addition to the reading difficulty 
level, other factors such as format, and nature and sequencing of information, may 
reduce the effectiveness of a job manual. 

The key, then, to the success or failure of the job manual or other aid is its design, 
which includes the nature and manner of presentation of the information in the aid. The 
foundation for the effective design of such materials is stimulus-response learning theory 
as applied to jobs by human factors scientists. In applying stimulus-response theory to 
job aid design, the job is viewed as being composed of a set of tasks in which the worker 
makes some approprie.te responses to a stimulus which cues that response. By performing 
a task analysis, the various cueing stimuli and their associated responses are identified. 
Decisions can then focus on determining which cueing stimuli are to be placed in the job 
aid, and how they will be displayed. Further details of the process of developing job 
performance aids are available in another report (17). 

The overall results of the review of job performance aids in REALISTIC suggests 
that the provision of more usable job material? may be the most rapid, least expensive 
method of assuring that job proficiency is attained by men across a wide spectrum of 
literacy or verbal aptitude skill levels. There is even some evidence (Chapter 2) to suggest 
that the use of printed materials may be facilitated more by the design characteristics of 
the materials than by increasing the reading skills of men, and this may be true for able 
as well as for poor readers. 

The fact that it is possible to design materials to permit margimally literate men to 
perform jobs satisfactorily indicates that it is feasible to hire the less literate man and to 
utilize him effectively, while providing liter? tcj' training to upgrade these skills. By this 
means it should be possible to employ th(; marginally Uterate in responsible, growth- 
oriented jobs, while maintaining the efficiency and effectiveness of the job subsystem. 
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Chapter 6 

SUMMARY AND MAJOR FINDINGS 

Work Unit REALISTIC was conducU.*d to (a) provide information concerninR the 
demands for reading, listening, and arithmetic skills in several major military oc-cupational 
specialties, and (b) to provide information and suggestions for reducing discrepancies 
between personnel literacy skill levels and the literacy skill levels required by the job. The 
four MOSs Studied-Armor Crewman, General Vehicle Mechanic, Unit and Organizational 
Supply Specialist, and Cook — have relatively high input of lower aptitude men, and hence 
literacy skills are of concern. 

In this report, data are presented which indicate relationships between reading, 
listening, and arithmetic skills of men in the cited MOSs and their job proficiency as 
measured by tests of proficiency on (a) job-related reading tasks, (b)job knowledge 
(paper-and-pencil) tests, (c) job performance (work sample) tests, and (d) supervisor 
ratings. In addition, data are presented concerning the reading difficulty level of printed 
materials in the Cook, Repairman, and Supply Specialist MOSs, and the extent of usage 
of these materials by men of different reading ability levels. Data are presented to 
indicate the extent to which men of differing reading levels ask questions of and listen to 
others (peers, supervisors) for job information. The extent and nature of use of arith- 
metic by men of differing reading ability levels is also discussed. The major findings on 
several categories of information are summarized below. 

(1) Information concerning the reading difficulty levels of materials in each MOS 

Reading materials for Repairmen and Supply Specialist exceeded the average 
reading ability levels of non-Category IV men by some four to six grade levels, and the 
reading levels of Category IV men by six to eight grade levels. The reading materials used 
by Cooks consisted of recipes primarily, and were written at a grade level comparable to 
the reading ability of men in both mental categories. 

Literature research indicated that much of the printed materials in the jobs 
studied could be redesigned to make them easier to use and comprehend. 

( 2) Information concerning the use of reading materials by men of different reading 
abilities in each MOS 

For Repairmen and Supply Specialists, it was found that the use of printed 
materials increased with increased skill in reading. Also, job sample test performance was 
better for Repairmen and Supply Specialists who used manuals. In the Cook job, skill in 
reading was unrelated to use of reading materials. This appeared to be related to the 
simple format and syntax of the Cook’s references which were cited by job incumbents. 

There was some evidence to suggest that the easier the printed material, the 
more it was likely to be used. Hence, increasing the use of reading materials might be 
accomplished both by redesign of materials and by remedial training in literacy. 

(3) Information concerning the use of listening sources on the job as a function of 
reading ability 

Supply Specialists and Repairmen who read at the 4-6.9 reading grade levels 
tended to rely as much (or perhaps a little more) on asking others for information as on 
reading for information, to a greater degree than men of higher reading levels. This was 
not true in the Cook job where the reading m.aterials were easiest. 

Non-Category IV and Category IV men were found to learn prose materials of a 
wide range of difficulty levels as well by listening as by reading. Also, a significant 
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to lt*arn by listt*ninK rathor than ri*adinj». 

Thi‘s«* finiiinys undersi-on- th*‘ important* of non-r*‘admj{ lan>,niaj{»* skills in job 
p»Tformanri‘. Listening as a oommunu ation skill is bt-inj; further studi.*d in a follow-up to 

KKALISTIC in HumKRO Work Unit LISTKN. 

(4) Information concerning the use of arithmetic on the job as a function of reading 

ability 

lnti*rvi€*ws with job incumtH^nts in thp Ropairman Supply Specialist, and C.ook 
jobs indicaU^d very little use of arithmetic. Where used in the Repairman and Supply 
Specialist jobs, however, this use was greater for higher reading ability men. This was not 

true for cooks. ... 

Lower reading level (4-6.9) men in the Repairman or Supply Specialist jobs 

tended to use mostly whole numbers or gauges in their arithmetic, while higher reading 
ability men used more complex arithmetic (fractions, decimals). In general, it seems as 
though men wdth better reading skills perform different, more responsible job duties, in 
which precision and accountability are demanded, than men with lower reading skills. 
Hence, they use more complicated arithmetic. However, no demand for skills beyond 
what is ordinarily taught at the sixth grade in school was found. 

(5) Information concerning reading ability and proficiency on. job related reading 



Performance of new accessions on job-related reading task tests for Repairman, 
Supply Specialist, and Cook MOSs correlated approximately .76 with performance on the 
standardized, grade school referenced reading test. This correlation was .65 for job 
reading task test and AFQT. Thus, the standardized reading test was a better predictor of 
proficiency on job-related materials than the AFQT was. 

(6) Information concerning reading, listening, and arithmetic skills and proficiency 
on Job Knowledge and Job Sample performance tests, ai\d superv sor ratings 

Only inconsequential relationships of any of the literacy variables to supervisor 
ratings were found for any MOS. The reading test score and /vFQT appeared to measure 
the same thing to a large dfjgree. Both predicted job proficiency equally well in all four 
MOSs. Thus, if the purpose of a co^itive test is simply to predict future job proficiency 
as measured by job knowledge or job sample scores, the AFQT is adequate. 

However, the Standardized Reading Test (SRT) predicted job knowledge and 
job sample performance about as well as the AFQT did. and, in addition, the SRT 
predicted performance on the job reading tasks. The S^'T thus out-performs the AFQT 
by predicting proficiency in a skill used in the job ^job-related reading. It also indicates 
whether a person exceeds some minimal cut-off point, and how far a man is below oi 
above some targeted reading level for different MOSs. 

Further, since the reading test provides information about skids which are 
directly amenable to improvement through training in a recognized content area (i.e., 
reading), whUe the AFQT does not lend itself to impro\ement through training in any 
recognized content area (i.e., there is no curriculum for teaching AFQT), consideration 
should be given to research to develop a reading test foi screening at the Armed Forces 
Examining and Entrance Stations. Such research should include an identification of the 
constraints upon military literacy training programs, and the identification of men of low 
literacy /language skills who would not be likely to succeed in the remedial programs 
operating under such constraints. 

Relationships of reading tj job proficiency .suggested a targeted reading level 
for Cooks of 7.0 grade level; for Repairmen and Armor Crewmen, 8.0 grade level; and for 
Supply Specialists, 9.0 grade level. These figures are based on the job printed materials 
and job training, procedures existing at the time of the study. If job materials and training 
procedures were considerably altered, the job proficiency of men reading below the 
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statt*d levels could be improved. However, if career advancement is desired, so that a man 
IS expected to assume a supervisor’s role, then remedial literacy training -at lea.st to the 
levels indicated— would be desirable to render general Army and Department of Defens<* 
publications useful to the man. 

Consistent with this view, it should be stated that, if men of lower literacy 
skills are accepted into service, then their later job security should not be contingent 
upon the acquisition of higher skill levels unless formal programs for acquiring such skills 
are available and emphasized during the early career period. This would seem to be of 
even greater importance if the manpower requirements are to be filled on the basis of a 
zero dependency on the draft. 

Arithmetic skill is highly related to reading and AFQT and predicts job 
knowledge and job performance about as well as these other indices. For selection or 
remedial training criterion purposes, a fifth or sixth grade level would seem sufficient. 

Listening and job knowledge were less highly related than reading and AFQT 
and job knowledge. However, listening was as highly related to job sample performance as 
were the reading test and AFQT. These findings affirm the importance of listening as a 
job-related communication skill. 
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Appendix A 

ANALYSIS OF VARIANCE TESTING-LISTENING VS. 
READERSHIP AS A FUNCTION OF READING ABILITY OF 
SUPPLY SPECIALISTS AND REPAIRMEN 



Source 


df 


Mean Square | 


F 


P 


Supply Specialist 












Between Subjects 












Reading Level (A) 




2 


2.385 


1.22 




Error (Subjects Within Groups) 


57 


1.956 






Within Subjects 












Listen vs. Read (B) 




1 


7.94 


14.18 


<.005 


A X B 




2 


5.56 


9.93“ 


<.005 


Error (B X Ss Within Groups) 




57 


.56 






Repairman 












Between Subjects 












Reading Level (A) 




2 


1.945 


.83 




Error (Subjects Within Groups) 


82 


2.352 






Within Subjects 












Listen vs. Read (B) 




1 


21.14 


12.81 


<.005 


A X B 




2 


10.57 


6.41“ 


<,005 


Error (B X Ss Within Groups) 




82 


1.65 







•Tests of simple effects of readership and reading level for Supply Specialist significant at 
p<.10 level; for Repairman, p<.025. Simple effects of listening by reading ability not significant 
in either MOS. 
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Appendix B 

STATISTICAL ADJUSTMENT OF PROFICIENCY MEASURES 
TO REMOVE EFFECTS OF JOB EXPERIENCE 



These data present the problem of extremely wide range of job experience in our 
subjects and substantial relationships between experience and job proficiency. The sam- 
ples contain men with from one month to more than 20 years of job experience. Average 
experience is 28 months while the median is only 13 months. In order to remove the 
linear effects of experience from the proficiency measures, as well as to reduce the 
problems of curvilinearity of regression and hcteroscedasticity, all job proficiency meas- 
ures were adjusted to remove the effects of experience. This consisted of: 

(1) Performing a logarithmic transformation on the experience (months on job) 

measures. 

(2) Calculating the linear regression of each proficiency measure on the log 
experience variable. 

(3) Determining the regression-based proficiency score predicted for each level 
of job experience. 

(4) Subtracting the estimated proficiency score due to experience from the 
individual’s raw proficiency score. 

The adjusted proficiency scores then are residual or deviation scores of individuals 
above or below that proficiency score attributable to work experience alone and thus 
have been freed of the linear effects of job exp>erience. 

As shown in Table B-1, er nerience is substantially related to raw scores on both the 
job knowledge and the performance proficiency measures, and slightly to supervisor 
ratings. The statistical adjustment reduced all correlations between experience and pro- 
ficiency to 0.00. Table B-2 contains the correlations between raw and adjusted pro- 
ficiency measures. 

By establishing, through subject selection, an equal number of high and low AFQT 
subjects at each level of experience, the UTILITY design foreordained the absence of any 
relationship between AFQT and experience in these data and thereby precluded the 
question of whether such a relationship does exist in this Army population. To the 
degree that literacy variables are related to AFQT, as shown in Table 9, the question of 
whether there is any relationship between literacy skills and overall job success as 
measured by retention and advancement must similatly remain moot to these data. Table 
B-3 displays the correlations between both the literacy variables and AFQT and experi- 
ence as obtained within the design limitations of this study. 
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Table B-1 



Correlations of Unadjusted Proficiency Measures 
With Log Months on Job 



MOS 


Proficiency Measure 


Knowledge 


Performance 


Supervisor Rating 


Armor Crewman 


.55 


.69 


.26 


Repairman 


.46 


.42 


.15 


Supply Specialist 


.64 


.44 


.14 


Cook 


.44 


.37 


.25 



Table B-2 



Correlations Between Unadjusted and Adjusted 
Job Proficiency Measures 



MOS 


Proficiency Measure 


Knowledge 


Performance 


Supervisor Hating 


Armor Crewman 


.83 


.72 


.97 


Repairman 


.90 


.91 


.99 


Supply Specialist 


.77 


.90 


.99 


Cook 


.90 


.93 


.97 



Table B-3 



Correlations Between Literacy Variables, AFQT, 
and Experience (Log Months on Job) 



MOS 


Reading 


Listening 


Arilhrr elic 


AFQT 


Armo: Crewman 


-.07 


-.04 


-.16 


.00 


Repairman 


.01 


.14 


-.08 


.02 


Supply Specialist 


.03 


.00 


-.04 


.01 


Cook 


-.04 


.07 


-.07 


.02 
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